Imaging apparatus and imaging method

ABSTRACT

Provided is an imaging apparatus including: an imaging unit; a display unit, through which at least a part of an imaging region of the imaging unit can be seen in a see-through manner and which is configured to display a stereoscopic image formed of a left eye image and a right eye image; a focal distance adjustment unit configured to adjust a focal distance of the imaging unit; and a display controller configured to generate the left eye image and the right eye image such that a display object indicating the focal distance is seen at a depth position corresponding to the focal distance and cause the display unit to display the left eye image and the right eye image.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Japanese Priority PatentApplication JP 2014-021833 filed Feb. 7, 2014, the entire contents ofwhich are incorporated herein by reference.

BACKGROUND

The present disclosure relates to an imaging apparatus and an imagingmethod and in particular to an imaging apparatus and an imaging methodby which a focal distance can be more easily known.

In recent years, a see-through head mounted display (HMD) has beendeveloped. For example, a see-through HMD that expresses a depth byadjusting a binocular disparity or convergence has been proposed (e.g.,see Japanese Patent Application Laid-open No. 2012-42654). Further,there has been proposed adding an imaging function to such a see-throughHMD for imaging a subject of a real space viewed by a user through thesee-through HMD.

SUMMARY

However, it is difficult for such a see-through HMD to check whether ornot focus is achieved on a desired subject in a captured image and thereis a fear that cumbersome processes are necessary for obtaining adesired imaging result.

In view of the above-mentioned circumstances, it is desirable to moreeasily know a focal distance.

According to an embodiment of the present technology, there is providedan imaging apparatus including: an imaging unit; a display unit, throughwhich at least a part of an imaging region of the imaging unit can beseen in a see-through manner and which is configured to display astereoscopic image formed of a left eye image and a right eye image; afocal distance adjustment unit configured to adjust a focal distance ofthe imaging unit; and a display controller configured to generate theleft eye image and the right eye image such that a display objectindicating the focal distance is seen at a depth position correspondingto the focal distance and cause the display unit to display the left eyeimage and the right eye image.

The display controller may be configured to set a size of the displayobject according to the focal distance.

The display unit may include a left eye image display unit configured todisplay the left eye image, and a right eye image display unitconfigured to display the right eye image.

The imaging apparatus may include a casing that is mounted on a head ofa user such that the left eye image display unit is positioned near infront of a left eye of the user and the right eye image display unit ispositioned near in front of a right eye of the user.

The display controller may be configured to cause the display unit tofurther display an image indicating an imaging field angle of theimaging unit.

The focal distance adjustment unit may be configured to be operated by auser to adjust the focal distance.

The imaging apparatus may further include a distance measurement unitconfigured to measure a distance to a subject, in which the focaldistance adjustment unit may be configured to adjust the focal distanceto the distance measured by the distance measurement unit.

The distance measurement unit may be configured to measure a distance tothe subject at a predetermined position in the imaging region.

The predetermined position may be a position in the imaging region,which is seen at a center of a display region of the display unit.

The imaging apparatus may further include a face detector configured todetect a face of the subject, in which the distance measurement unit maybe configured to measure a distance to the face of the subject detectedby the face detector.

The imaging apparatus may further include a line-of-sight detectorconfigured to detect a line-of-sight direction of the user, in which thedistance measurement unit may be configured to measure a distance to thesubject in the line-of-sight direction of the user detected by theline-of-sight detector.

The imaging apparatus may further include an instruction reception unitconfigured to receive an execution instruction of measurement of thedistance to the subject, in which the distance measurement unit may beconfigured to measure the distance to the subject based on the executioninstruction received by the instruction reception unit.

According to another embodiment of the present technology, there isprovided an imaging method including: adjusting a focal distance of animaging unit; generating a left eye image and a right eye image suchthat a display object indicating the focal distance is seen at a depthposition corresponding to the focal distance and causing a display unit,through which at least a part of an imaging region of the imaging unitcan be seen in a see-through manner and which is configured to display astereoscopic image formed of the left eye image and the right eye image,to display the left eye image and the right eye image; and imaging asubject by the imaging unit.

In the embodiments of the present technology, a focal distance of animaging unit is adjusted. A left eye image and a right eye image aregenerated such that a display object indicating the focal distance isseen at a depth position corresponding to the focal distance and theleft eye image and the right eye image are displayed on a display unit,through which at least a part of an imaging region of the imaging unitcan be seen in a see-through manner and which is configured to display astereoscopic image formed of the left eye image and the right eye image.A subject is imaged by the imaging unit.

According to the embodiments of the present technology, it is possibleto image a subject. Further, according to the embodiments of the presenttechnology, it is possible to more easily know a focal distance.

These and other objects, features and advantages of the presentdisclosure will become more apparent in light of the following detaileddescription of best mode embodiments thereof, as illustrated in theaccompanying drawings.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a view showing an example of an outer appearance of asee-through HMD;

FIG. 2 is a block diagram showing a main configuration example of thesee-through HMD;

FIG. 3 is a flowchart showing an example of a flow of imaging.

FIG. 4 is a view showing a display example of a focus icon;

FIG. 5 is a view showing a display example of a focus icon;

FIG. 6 is a view showing a display example of a focus icon;

FIG. 7 is a view explaining how to image as an example; and

FIG. 8 is a view explaining how to image as another example.

DETAILED DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments for carrying out the present disclosure(hereinafter, referred to as embodiments) will be described. Note thatdescriptions thereof will be given in the following order.

1. First Embodiment (Imaging Apparatus)

2. Second Embodiment (See-Through HMD)

<1. First Embodiment>

<Checking Imaging Result by See-Through Display>

In recent years, a see-through head mounted display (HMD) has beendeveloped. The see-through HMD uses a display unit (see-through display)that permits light from a rear side to pass such that a landscape of areal space on the rear side of the display unit can be seen in asee-through manner. With this, a user of the see-through HMD can see animage displayed on the see-through display in an overlapping state withthe landscape of the real space on the rear side of the see-throughdisplay (hereinafter, also referred to as see-through picture).

There has been proposed adding an imaging function (camera) to such asee-through HMD for imaging a subject of a real space viewed by the userthrough the see-through HMD (in other words, imaging apparatus withsee-through display). For example, there has been proposed a use casewhere the user who wears the see-through HMD having the imaging function(camera) takes a picture of a subject by the camera of the HMD as a lifelog and share the obtained captured image over a network.

By the way, from the past, for checking whether or not focus is achievedon a desired subject when taking a picture of the subject, the user hadto check an imaging result (captured image). The captured image can bechecked by, for example, the see-through HMD or another device.

For example, in the case where a captured image is displayed and checkedon a display unit of another device such as a smart phone, the user hasto prepare the other device and transfer data of the captured image fromthe see-through HMD to the other device. The cumbersome processes arenecessary only for checking the captured image.

Further, for example, in the case where a captured image is displayedand checked on the see-through display of the see-through HMD, the userhas to check the captured image in an overlapping state with a landscapeof the real space (see-through picture). Thus, it is difficult toclearly view the captured image and there is a fear that a small error,for example, focus deviation of the captured image may be overlooked.

As described above, for example, in the case of the imaging apparatuswith the see-through display like the see-through HMD, it is notpossible to easily check a focal distance of a captured image (checkwhether or not focus is achieved on a desired subject, for example).Thus, an imaging failure, for example, focus deviation (state in whichfocus is not achieved on the desired subject) easily occurs. Therefore,cumbersome processes including checking an image, re-taking the image,and the like are necessary for obtaining a desired imaging result.

<Imaging Apparatus>

In view of this, there are provided an imaging unit, a display unit, afocal distance adjustment unit, and a display controller. Through thedisplay unit, at least a part of an imaging region of the imaging unitcan be seen in a see-through manner. The display unit is configured todisplay a stereoscopic image formed of a left eye image and a right eyeimage. The focal distance adjustment unit is configured to adjust afocal distance of the imaging unit. The display controller is configuredto generate the left eye image and the right eye image such that adisplay object indicating the focal distance is seen at a depth positioncorresponding to the focal distance and cause the display unit todisplay the left eye image and the right eye image.

That is, the imaging apparatus adjusts the focal distance of the imagingunit and generates the left eye image and the right eye image of thestereoscopic image such that the display object indicating the focaldistance is seen at the depth position corresponding to the focaldistance. The left eye image and the right eye image are displayed onthe display unit through which the at least part of the imaging regionof the imaging unit can be seen in a see-through manner and which isconfigured to display the stereoscopic image formed of the left eyeimage and the right eye image. Then, a subject is imaged by the imagingunit.

That is, the display object indicating the focal distance (hereinafter,also referred to as focus icon) is displayed on the display unit(see-through display) as the stereoscopic image.

The stereoscopic image is an image that looks three-dimensional to auser who views the image and recognizes a positional relationship in adepth direction (depth feeling). The stereoscopic image is formed of,for example, images with a plurality of viewpoints. Disparity andconvergence angle between the images express the depth feeling. Thestereoscopic image is, for example, formed of a left eye image and aright eye image mainly viewed by the left and right eyes of the user,respectively.

The focus icon that is such a stereoscopic image is displayed on thesee-through display. The position of the displayed focus icon in thedepth direction (hereinafter, also referred to as depth position)expresses the focal distance.

With this configuration, the user can easily check the focal distancebased on the depth position of the focus icon displayed on thesee-through display. For example, the user can easily check whether ornot focus is achieved on a desired subject by judging whether or not thedepth position of the focus icon coincides with a depth position of thesubject of the real space in the see-through picture. Thus, the imagingapparatus is capable of reducing imaging failures, for example, focusdeviation (state in which focus is not achieved on the desired subject).Consequently, the user can more easily obtain a desired imaging result.

Note that the focus icon can have any shape. For example, the focus iconmay have a cross, rectangular, or circular shape or may have anothershape. Further, a single focus icon may be provided or a plurality offocus icons may be provided.

The display controller of the imaging apparatus may be configured to seta size of the display object according to the focal distance. Forexample, the size of the focus icon may be reduced as the focal distanceincreases (in other words, the size of the focus icon may be increasedas the focal distance decreases). That is, the focal distance may beexpressed using the depth position and size of the focus icon. With thisconfiguration, the user can check the focal distance more intuitivelyand more accurately (e.g., the user can check whether or not focus isachieved on a desired subject more easily and more accurately). Thus,the imaging apparatus is capable of reducing imaging failures, forexample, focus deviation (state in which focus is not achieved on thedesired subject). Consequently, the user can more easily obtain adesired imaging result.

Further, the display unit of the imaging apparatus may include a lefteye image display unit configured to display the left eye image and aright eye image display unit configured to display the right eye image.With this configuration, the user views the left eye image display unitwith the left eye and views the right eye image display unit with theright eye, to thereby be able to view the focus icon as the stereoscopicimage (with depth feeling). That is, the imaging apparatus is capable ofproviding the user with the stereoscopic image without needingpolarizing spectacles or the like.

The imaging apparatus may include a casing that is mounted on a head ofa user such that the left eye image display unit is positioned near infront of a left eye of the user and the right eye image display unit ispositioned near in front of a right eye of the user. That is, theimaging apparatus can be embodied as a so-called HMD (see-through HMD).

The display controller of the imaging apparatus may be configured tocause the display unit to further display an image indicating an imagingfield angle of the imaging unit. With this configuration, the user caneasily know a range of an imaging region (range of captured image). Notethat the image can be seen at any distance in the depth direction. Forexample, the image may be seen in the forefront. Alternatively, thisimage does not need to produce a depth feeling (does not need to be astereoscopic image).

The focal distance adjustment unit of the imaging apparatus may beconfigured to be operated by a user to adjust the focal distance. Thatis, the focal distance may be adjusted according to a user's operation(so-called manual focus). Specifically, in this case, for example, whenthe user manually adjusts the focal distance, the depth position of thefocus icon is changed according to the focal distance changed by theoperation. By performing an operation while viewing the depth positionof the focus icon, the user can adjust the focal distance while checkingthe focal distance. Thus, the user can adjust the focal distance to adesired depth position more easily and more accurately.

The imaging apparatus may further include a distance measurement unitconfigured to measure a distance to the subject, in which the focaldistance adjustment unit may be configured to adjust the focal distanceto the distance measured by the distance measurement unit. That is, thefocal distance may be adjusted by the imaging apparatus (so-calledautofocus). In this case, the user can check the focal distance byviewing the depth position of the focus icon. That is, the user cancheck an autofocus result (e.g., whether or not focus is achieved at adesired distance) more easily and accurately.

The distance measurement unit of the imaging apparatus may be configuredto measure a distance to the subject at a predetermined position in theimaging region. For example, the predetermined position may be aposition in the imaging region, which is seen at a center of the displayregion of the display unit. Of course, the predetermined position may beany position as long as it is defined in advance. The predeterminedposition is not limited to the position in the imaging region, which isseen at the center of the display region of the display unit. Thus, theposition of the subject in the display region, whose distance is to bemeasured, is fixed, and hence the user can easily set the subject as thedistance measurement target by adjusting the attitude (position,orientation, etc.) of the imaging apparatus such that the desiredsubject is located at that position (e.g., center of display region).

The imaging apparatus may further include a face detector configured todetect a face of the subject in which the distance measurement unit maybe configured to measure a distance to the face of the subject detectedby the face detector. With this configuration, focus can be easilyachieved on the face of the subject.

The imaging apparatus may further include a line-of-sight detectorconfigured to detect a line-of-sight direction of the user. The distancemeasurement unit may be configured to measure a distance to the subjectin the line-of-sight direction of the user detected by the line-of-sightdetector. With this configuration, the user can easily set the subjectas the distance measurement target only by directing the line-of-sightto the desired subject.

The imaging apparatus may further include an instruction reception unitconfigured to receive an execution instruction of measurement of thedistance to the subject. The distance measurement unit may be configuredto measure the distance to the subject based on the executioninstruction received by the instruction reception unit. With thisconfiguration, only if the user issues an instruction, the distancemeasurement (i.e., also focal distance adjustment) can be performed.That is, the focal distance can be fixed unless the user issues aninstruction. With this, for example, the user can change the imagingdirection and the like while maintaining the focal distance after thefocal distance is adjusted.

Note that the above-mentioned display object indicating the focaldistance may be displayed on a non-see-through display unit that doesnot permits light to pass. For example, the non-see-through display unitmay display a display object (focus icon) with the display object (focusicon) being overlapped on a captured image (also referred to asthrough-image) obtained by the imaging unit. Also in this case, as inthe case where the display object is displayed on the see-throughdisplay unit as described above, it is only necessary to generate a lefteye image and a right eye image and cause the display unit to displaythem such that the display object is seen at a depth positioncorresponding to a focal distance.

That is, the above-mentioned phrase “seeing at least a part of theimaging region of the imaging unit in a see-through manner” includes notonly “seeing the landscape of the real space on the rear side of thedisplay through the display” as in the see-through display but also“seeing the through-image obtained by the imaging unit and displayed onthe display” as in the above-mentioned non-see-through display.

<2. Second Embodiment>

Outer Appearance of See-Through HMD

The present technology is also applicable to an apparatus other than theimaging apparatus. For example, the present technology may be applied tothe see-through HMD. FIG. 1 is a view showing an example of an outerappearance of a see-through HMD as an embodiment of the imagingapparatus to which the present technology is applied. For example, asshown in “A” of FIG. 1, a casing 111 of a see-through HMD 100 has aso-called eyeglass shape. Like eyeglasses, the see-through HMD 100 isput on a face of the user with ends of the casing 111 resting on ears ofthe user and used.

The portions corresponding to lenses of the eyeglasses are display units112 (right eye display unit 112A and left eye display unit 112B). Whenthe user wears the see-through HMD 100, the right eye display unit 112Ais positioned near in front of the right eye of the user and the lefteye display unit 112B is positioned near in front of the left eye of theuser.

The display unit 112 is a see-through display that permits light topass. Thus, the right eye of the user can see, through the right eyedisplay unit 112A, a rear side thereof, that is, the landscape of thereal space (see-through picture) in front of the right eye display unit112A. Similarly, the left eye of the user can see, through the left eyedisplay unit 112B, a rear side thereof, that is, the landscape of thereal space (see-through picture) in front of the left eye display unit112B. Thus, the user see an image displayed on the display unit 112while the image overlapping on the landscape of the real space in frontof the display unit 112.

The right eye display unit 112A displays an image (right eye image) forthe right eye of the user. The left eye display unit 112B displays animage (left eye image) for the left eye of the user. That is, thedisplay unit 112 is capable of displaying different images to the righteye display unit 112A and the left eye display unit 112B and displayinga stereoscopic image, for example.

The stereoscopic image is an image that is formed of the right eye imageand the left eye image both having binocular disparity and convergenceangle and looks far or close from/to the user by controlling thebinocular disparity and convergence angle. That is, it is possible tocontrol the depth position of the stereoscopic image. Note that thedepth position of the stereoscopic image is not a position at which theimage is actually displayed but a virtual position at which the image isseen by the user as if it is present (also referred to as virtual imageposition).

That is, the display unit 112 is capable of displaying an image(stereoscopic image) such that the user can see the image as if it ispresent in the real space in front of the display unit 112 as viewedfrom the user.

Further, as shown in FIG. 1, a hole 113 is provided near the displayunit 112 of the casing 111. Inside the casing 111 near the hole 113,there is provided an imaging unit that images a subject. The imagingunit images through the hole 113 a subject of the real space in front ofthe see-through HMD 100 (in front of the see-through HMD 100 as viewedby the user who wears the see-through HMD 100). More specifically, theimaging unit images a subject of the real space, which is positioned inthe display region of the display unit 112 (right eye display unit 112Aand left eye display unit 112B) as viewed from the user. With this,image data of the captured image is generated. The generated image datais, for example, stored in a predetermined storage medium or transmittedto another device.

Note that the hole 113 (i.e., imaging unit) can be located at anyposition and may be located at a position other than that of the exampleshown in “A” of FIG. 1. Further, any number of holes 113 (i.e., imagingunits) can be provided. Only one hole 113 (i.e., imaging unit) may beprovided as shown in “A” of FIG. 1 or a plurality of holes 113 (i.e.,imaging units) may be provided.

Further, the casing 111 can have any shape as long as it can be mountedon the face (head) of the user such that the right eye display unit 112Ais positioned near in front of the right eye of the user and the lefteye display unit 112B is positioned near in front of the left eye of theuser. For example, the see-through HMD 100 may have a shape as shown in“B” of FIG. 1.

In the case of the example of “B” of FIG. 1, a casing 131 of thesee-through HMD 100 is formed in a shape sandwiching and fixing the headof the user from the back. Also the display unit 132 in this case is asee-through display similar to the display unit 112. That is, a displayunit 132 also includes a right eye display unit 132A and a left eyedisplay unit 132B. When the user wears the see-through HMD 100, theright eye display unit 132A is positioned near in front of the right eyeof the user and the left eye display unit 132B is positioned near infront of the left eye of the user.

The right eye display unit 132A is a display unit similar to the righteye display unit 112A. The left eye display unit 132B is a display unitsimilar to the left eye display unit 112B. That is, the display unit 132is also capable of displaying a stereoscopic image like the display unit112.

Further, also in the case of “B” of FIG. 1, a hole 133 similar to thehole 113 is provided near the display unit 132 of the casing 131 as inthe case of “A” of FIG. 1. Inside the casing 131 near the hole 133,there is provided an imaging unit that images a subject. The imagingunit images through the hole 133 a subject of the real space in front ofthe see-through HMD 100 (in front of the see-through HMD 100 as viewedby the user who wears the see-through HMD 100) as in the case of “A” ofFIG. 1.

Of course, the hole 133 (i.e., imaging unit) can be located at anyposition as in the case of “A” of FIG. 1 and may be located at aposition other than that of the example shown in “B” of FIG. 1. Further,any number of holes 133 (i.e., imaging units) can be provided as in thecase of “A” of FIG. 1.

Further, as in the example shown in “C” of FIG. 1, some of thecomponents of the see-through HMD in the example of “A” of FIG. 1 may beconfigured as those separate from the casing 111. In the case of theexample of “C” of FIG. 1, the casing 111 is connected to a control box152 via a cable 151.

The cable 151 is a communication path for predetermined wiredcommunication and electrically connected to a circuit inside the casing111 and a circuit inside the control box 152. The control box 152includes some of the components (circuit, etc.) inside the casing 111 inthe case of the example of “A” of FIG. 1. For example, the control box152 may include a controller and a storage unit or the like that storesimage data. The circuit inside the casing 111 may communicate with thecircuit inside the control box 152. The imaging unit inside the casing111 may perform imaging under the control of the controller of thecontrol box 152. The image data of the captured image obtained byimaging may be supplied to the control box 152 and stored in the storageunit.

The control box 152 can be put in a pocket or the like of the clothes ofthe user, for example. With this configuration, the casing 111 of thesee-through HMD 100 can be downsized in comparison with the case of “A”of FIG. 1.

Note that the communication between the circuit inside the casing 111and the circuit inside the control box 152 may be wired or wireless. Inthe case of the wireless communication, the cable 151 can be omitted.

<Inner Configuration Example>

FIG. 2 is a block diagram showing an inner configuration example of thesee-through HMD 100. As shown in FIG. 2, the see-through HMD 100includes a system controller 211.

The system controller 211 is, for example, configured by a microcomputer including a central processing unit (CPU), a read only memory(ROM), a random access memory (RAM), a non-volatile memory unit, aninterface, and the like and controls the respective sections of thesee-through HMD 100. The system controller 211 controls the respectivesections based on an inner operation program. For example, the systemcontroller 211 controls the respective sections and performs displaycontrol on an image to be displayed on the display unit 112.

Further, the see-through HMD 100 includes a sensor unit 221, a sensorcontroller 222, and a sensor signal processor 223.

The sensor unit 221 includes an arbitrary sensor such as an accelerationsensor, gyro sensor, magnetic sensor, and atmospheric pressure sensormounted near the display unit 112. The sensor unit 221 detects, forexample, a motion of the head of the user, a motion of the neck, or amotion of the see-through HMD 100 as a signal corresponding to a motionof the user. Further, the sensor unit 221 includes a capacitive sensor,a button, a global positioning system (GPS), and the like. A signal of asensor system used by the user for operating the see-through HMD 100 mayalso be processed by this section.

That is, the sensor unit 221 includes an arbitrary input device andreceives information input via the input device. The input device can belocated at any position and not limited to be positioned in the vicinityof the display unit 112.

Further, the sensor unit 221 may include a sensor that detects theline-of-sight of the user or the like. For example, as this sensor, thesensor unit 221 may be placed in vicinity of the display unit 112 andinclude an imaging unit (line-of-sight imaging unit) that images an eyeregion of the user. In this case, for example, the system controller 211may analyze the image of the eye region of the user that is captured bythe line-of-sight imaging unit, to thereby detect a line-of-sightdirection, a focal distance, a degree of opening of pupils, a funduspattern, opening/closing of eyelids, and the like. Further, instead ofsuch a line-of-sight imaging unit, the sensor unit 221 includes a lightemitting unit that is positioned near the display unit 112 and emitslight to the eye region of the user and a light receiving unit thatreceives and photoelectrically converts the reflection light from theeye region and outputs it as an electrical signal (light receivingsignal). In this case, the system controller 211 may detect theline-of-sight direction or the like based on the light receiving signalobtained by the light receiving unit. Alternatively, for example, thesystem controller 211 may detect a thickness of a crystalline lens ofthe user based on the light receiving signal obtained by converting thereflection light received by the light receiving unit into theelectrical signal. That is, the sensor unit 221 may include aline-of-sight detector that detects a line-of-sight direction of theuser. Any detection method can be used.

The sensor controller 222 performs sensor control regarding, forexample, which sensor is driven at which timing or by which drivingmethod the detection is performed based on an instruction from thesystem controller 211. Further, the sensor signal processor 223 subjectsa sensor signal detected by the sensor unit 221 to various types ofnumerical processing such as averaging and variance calculation aspre-processing before it is output to the system controller 211.

The see-through HMD 100 needs in some cases the user's operation forpower on/off, start/end of display of various information images, changein image content, display adjustment of luminance, hue, and the like,change in the display region on the display screen, and the like. Forexample, an operating element as an operation key or operation dialoperated by the user and an operation detecting mechanism that detectsan operation of the operating element may be provided, for example, asthe sensor unit 221 for the operations (detection of trigger forprocessing operation) such that the system controller 211 can detect theuser's operation.

Instead of the provision of such an operating element, the systemcontroller 211 may judge a user's operation intention and suitableoperation processing based on the situation of the user that is detectedby the sensors of the sensor unit 221 (eye motion, motion and state ofbody, etc.), and may perform the corresponding processing.

In addition, the sensor unit 221 may be configured to detect otherexternal world information (e.g., detection information on thesurroundings of the see-through HMD 100, location, date and time, andstate of the subject). The system controller 211 may determine suitableoperation processing based on the external world information detected bythe sensor unit 221 and perform the determined processing.

Further, the see-through HMD 100 includes an image generator 231, adisplay image processor 232, a display drive unit 233, and a displaycontroller 234.

The image generator 231 generates an image signal under the control ofthe system controller 211. The system controller 211 causes the imagegenerator 231 to generate an image signal that becomes an image to bepresented to the user according to the content and numerical valuesobtained from the respective sections and generates an image signalserving as a picture image, a graph image, a literal image, or an imagefor giving an alarm to the user.

The display image processor 232 includes a so-called video processor andsubjects the supplied image signal to various types of displayprocessing. For example, the display image processor 232 performsluminance adjustment, color correction, contrast adjustment, sharpness(outline emphasis) adjustment, and the like of the imaging signalsupplied from the imaging signal processor 252. Additionally, thedisplay image processor 232 may set a display position on the screen.Still additionally, the display image processor 232 may performgeneration of an enlarged image in which an imaging signal is partiallyenlarged or generation of a reduced image, image effect processing suchas soft focus, mosaic, luminance inversion, highlight display(highlighting) of a part of an image, and change in entire coloratmosphere, separation or combination of an image for division displayof a captured image, generation of a character image or picture image,processing of combining the generated image with the captured image, andthe like. The display image processor 232 supplies the processed imagesignal to the display drive unit 233.

Note that the display image processor 232 may perform such processingaccording to an instruction supplied from the display controller 234 ora sensor signal supplied from the sensor signal processor 223.Alternatively, the display image processor 232 may perform the sameprocessing also on an image supplied from the display controller 234.For example, although the display image processor 232 is supplied withan image signal generated by the image generator 231, the display imageprocessor 232 may perform predetermined signal processing for displayalso on this image signal and supplies it to the display drive unit 233.

The display drive unit 233 is configured by a pixel driving circuit fordisplaying an image signal supplied from the display image processor 232on the display unit 112. That is, the display drive unit 233 applies adriving signal based on a video signal at a predeterminedhorizontal/vertical driving timing on each of pixels arranged in amatrix form in the display unit 112, and causes them to perform display.Alternatively, the display drive unit 233 may also control the luminanceof each pixel of the display unit 112 and put the entire screen and apart of the screen in a through-state.

The display controller 234 controls, based on an instruction of thesystem controller 211, a processing operation of the display imageprocessor 232, an operation of a display driving unit, and an imagedisplayed on each of the left and right display units, and instructs tothe display image processor 232 to process a signal. Further, thedisplay controller 234 controls the display drive unit 233 to switchbetween a through-state, an image display state, a single-eye displaystate, for example.

Further, the see-through HMD 100 includes an audio generator 241, anaudio output unit 242, an audio input unit 243, and an audio signalprocessor 244.

According to an instruction of the system controller 211, the audiogenerator 241 generates an audio signal of a voice message or the likeby voice synthesis processing or generates another audio signal to bepresented to the user, for example, electronic sound.

The audio output unit 242 includes, for example, a speaker or earphonespeaker mounted on the see-through HMD 100 and an amplifier circuit forthe speaker. By an audio signal generated by the audio generator 241being supplied to the audio output unit 242, the user can listen to avoice message, electronic sound, or the like. Note that the audio outputunit 242 may be configured as a so-called bone conduction speaker.

The audio input unit 243 includes a microphone amplifier unit thatamplifies an audio signal obtained by a microphone and an A/D converterand outputs audio data.

The audio signal processor 244 includes, for example, a digital signalprocessor and a D/A converter. The audio data obtained by the audioinput unit 243 and the audio data generated by the audio generator 241are supplied to the audio signal processor 244. The audio signalprocessor 244 performs processing, for example, volume control, soundquality control, or production of acoustic effects on the supplied audiodata according to the control of the system controller 211. Then, theaudio signal processor 244 converts the processed audio data into ananalog signal and supplies it to the audio output unit 242. Note thatthe audio signal processor 244 is not limited to the configuration ofperforming digital signal processing and may process a signal through ananalog amplifier and an analog filter.

The audio signal output from the audio signal processor 244 is outputfrom the earphone speaker of the audio output unit 242 as audio. Withsuch a configuration, the user can hear external sound collected by theaudio input unit 243 and listen to sound generated by the audiogenerator 241.

Further, the see-through HMD 100 includes an imaging unit 251, animaging signal processor 252, and an imaging controller 253.

The imaging unit 251 is provided with a lens system constituted of animaging lens, a diaphragm, a zoom lens, a focus lens, and the like, adriving system for causing the lens system to perform a focus operationand a zoom operation, a solid-state imaging element array that detectsimaging light obtained by the lens system and photoelectrically convertsthe imaging light into an imaging signal. The solid-state imagingelement array is formed of, for example, a charge coupled device (CCD)sensor array and a complementary metal oxide semiconductor (CMOS) sensorarray.

The imaging unit 251 images, for example, a landscape in front of theuser (subject of real space in front of user) via the hole 113. Ofcourse, the imaging unit 251 may image a landscape in another direction,for example, on the rear side of the user.

The imaging signal processor 252 includes a sample hold/automatic gaincontrol (AGC) circuit, a video A/D converter, or the like that performsgain control or waveform shaping on a signal obtained by the solid-stateimaging element of the imaging unit 251 and obtains an imaging signal asdigital data. Alternatively, the imaging signal processor 252 mayperform white balance processing, luminance processing, color signalprocessing, blur correction processing, and the like on the imagingsignal.

The imaging controller 253 controls, based on an instruction from thesystem controller 211, operations of the imaging unit 251 and theimaging signal processor 252. For example, the imaging controller 253controls on/off of operations of the imaging unit 251 and the imagingsignal processor 252. Further, the imaging controller 253 performscontrol (motor control) for causing the imaging unit 251 to performoperations of autofocus, automatic exposure control, diaphragmadjustment, zoom, focus change, and the like.

Note that, if a movable mechanism capable of changing a subjectdirection using the imaging lens is provided, the imaging controller 253controls the operation of the movable mechanism to change the directionof the imaging lens in the imaging unit 251 based on an instruction ofthe system controller 211.

Further, the imaging controller 253 includes a timing generator andcontrols, according to a timing signal generated by the timinggenerator, signal processing operations of the solid-state imagingelement of the imaging unit 251 and the sample hold/AGC circuit andvideo A/D converter of the imaging signal processor 252. Further,variable control of the imaging frame rate can also be performed owingto this timing control.

In addition, the imaging controller 253 may control imaging sensitivityor signal processing in the solid-state imaging element of the imagingunit 251 and the imaging signal processor 252. For example, as theimaging sensitivity control, the imaging controller 253 may perform gaincontrol of a signal read from the solid-state imaging element of theimaging unit 251, black level setting control, various types ofcoefficient control of imaging signal processing in a digital dataphase, correction amount control in blur correction processing, and thelike.

Further, the imaging controller 253 may control entire sensitivityadjustment not considering a wavelength band, a sensitivity adjustmentin which the imaging sensitivity of a particular wavelength band, forexample, an infrared region or a ultraviolet region is adjusted (e.g.,imaging to cut particular wavelength band), or the like. For example,the sensitivity adjustment based on the wavelength can be performed byinsertion of a wavelength filter in an imaging lens system or wavelengthfilter calculation processing with respect to an imaging signal. Inthese cases, the imaging controller 253 is capable of controlling thesensitivity by controlling the insertion of the wavelength filter,designating the filter calculation coefficient, or the like.

For example, a captured image signal obtained by the imaging unit 251and the imaging signal processor 252 is supplied to the display imageprocessor 232 together with an information image signal generated by theimage generator 231. The display image processor 232 performs theabove-mentioned various types of signal processing on each image signal.Further, the display image processor 232 subjects two image signals tosignal processing (image combination processing) as the screen divisionfor causing the display unit 112 to display the two image signals at thesame time.

The image signals combined by the display image processor 232 issupplied to the display drive unit 233 and displayed on the display unit112, and thus the captured image and other images are displayed on thedisplay unit 112 at the same time. In other words, the user can viewvarious images also while viewing the captured image.

There is a case where a user's operation is necessary for start/end ofan imaging operation, a zoom operation, a focus operation, a capturedimage adjustment, and the like. Of course, there is also a case where auser's operation is necessary for power on/off, start/end of display ofvarious information images, change in image content, display adjustmentof luminance, hue, and the like, change of the display region on thedisplay screen, and the like. For example, the operating element such asthe operation key for the operation (operation trigger) may be providedin the sensor unit 221. Alternatively, the system controller 211 mayjudge a user's operation intention and suitable operation processingbased on the situation of the user that is detected by various sensorsof the sensor unit 221 (eye motion, motion and state of body, etc.) andperform the corresponding processing.

In addition, the sensor unit 221 may be configured to be capable ofdetecting external world information (detection information ofsurroundings of see-through HMD 100, location, date and time, situationof subject, etc.) and may determine suitable operation processing basedon the external world information and perform the determined processing.

Further, the see-through HMD 100 includes a storage unit 261, acommunication unit 262, and a drive 263.

The storage unit 261 includes a hard disk drive (HDD), a solid-statememory such as a flash memory, a memory card installing a solid-statememory, and an arbitrary storage medium such as an optical disc, anopto-magnetic disc, and a hologram memory and records/reproduces data onthe storage medium.

For example, image data serving as an imaging signal captured by theimaging unit 251 and processed by the imaging signal processor 252,image data received by the communication unit 262, and image signals ofvarious types of information generated by the image generator 231 may bestored in the storage unit 261. Alternatively, audio data obtained inthe audio input unit 243, audio data received by the communication unit262, and audio data generated by the audio generator 241 may also bestored in the storage unit 261.

The storage unit 261 encodes the supplied image data or audio data forstoring it in the storage medium under the control of the systemcontroller 211 and stores the encoded data to the storage medium. Thestorage unit 261 reproduces the image data or audio data and outputs itto another processor from the storage medium under the control of thesystem controller 211.

The data reproduced by the storage unit 261 can include any types ofdata to be display targets. For example, the data reproduced by thestorage unit 261 is moving image content such as movie and a video clip,still image content captured by a digital still camera or the like andstored in the storage medium, data of an electronic book or the like,computer use data such as image data, text data, and table calculationdata created by the user using a personal computer or the like, or agame image.

The communication unit 262 transmits and receives data to/from anapparatus outside the see-through HMD 100 (hereinafter, also referred toas external apparatus). As the external apparatus, there are exemplifieda video camera having a communication function, an imaging apparatussuch as a digital still camera, a computer apparatus, a smart phone, asmart watch, an AV apparatus such as a video storage apparatus and atelevision receiver, and a network server apparatus.

Further, for example, the communication unit 262 may be configured toperform network communication with a network access point viashort-distance wireless communication by using a predetermined systemsuch as a wireless local area network (LAN) and Bluetooth (registeredtrademark) or may directly perform wireless communication with theexternal apparatus having the corresponding communication function.

The data transmitted from the external apparatus to the see-through HMD100 can include any types of data to be display targets. For example, ifthe external apparatus serves as an imaging apparatus, the datatransmitted from the external apparatus to the see-through HMD 100 isimage data captured by the imaging apparatus. If the external apparatusserves as a content source apparatus, the data transmitted from theexternal apparatus to the see-through HMD 100 is, for example, movingimage content such as a movie and a video clip, still image contentcaptured by a digital still camera or the like and stored in a storagemedium, data of an electronic book or the like, computer use data suchas image data, text data, and table calculation data created by the userusing a personal computer or the like, or a game image.

Further, audio data obtained by the audio input unit 243, audio datareproduced by the storage unit, and audio data received by thecommunication unit 262 are supplied to an audio signal processoraccording to an instruction of the system controller 211.

Therefore, while wearing the device, the user can view and listen to acaptured image and external sound recorded when the image is captured,view and listen to an image and audio reproduced by the storage unit, orview and listen to an image and audio received by the communicationunit.

In particular, by an image of the image generator 231 being supplied tothe display image processor together with the captured image, thereproduced image, or the received image, various information images aredisplayed together with the captured image, the reproduced image, or thereceived image.

At a timing at which audio data is generated by the audio generator 241,by the generated audio data being supplied to the audio signal processor244, the user can listen to a voice message, alarm sound, or the likegenerated by the audio generator 241 while listening to the externalsound, the reproduced sound, or the received sound, for example.

In addition to the operations of the display system and the operationsrelated to the imaging function, the system controller has to determinetriggers for operation control for play, cueing,fast-forwarding/rewinding, pause, recording, and the like in the storageunit 261 and operation control related to the transmission and receptionin the communication unit 262. Also in this case, an operating elementsuch as an operation key operated by the user may be provided in, forexample, the sensor unit 221 such that processing corresponding to anoperation performed on the operating element can be performed. Further,the system controller 211 may determine a user's operation intention andsuitable operation processing based on the situation of the user that isdetected by the sensor unit 221 (eye motion, motion and state of body,etc.) and perform the corresponding processing.

In addition, the sensor unit 221 may be configured to be capable ofdetecting external world information of the see-through HMD 100(detection information of surroundings of display apparatus, location,date and time, situation of subject, etc.), and may determine suitableoperation processing based on the external world information and performthe determined processing.

A removable medium 264 such as an optical disc and semiconductor memoryis appropriately mounted on the drive 263. Computer programs and dataare read from the removable medium 264 by the drive 263, supplied to thesystem controller 211, and stored or installed in the storage unit 261.

In the see-through HMD 100 having the above-mentioned configuration, asdescribed above, the display unit 112 permits light on the rear side ofthe display unit 112 (in front of the display unit 112 as viewed fromthe user) to pass, and displays a stereoscopic image formed of imageshaving a plurality of viewpoints in an overlapping state on thelandscape of the real space on the rear side.

Further, the imaging unit 251 images the subject of the real spacepositioned in the display region of the display unit 112 as viewed fromthe user.

The imaging unit 251 has a focal distance adjustment function (focusfunction). For example, the imaging unit 251 may have a manual focaldistance adjustment function of adjusting the focal distance accordingto a user's operation (also referred to as manual focus).

The image generator 231 generates a stereoscopic image indicating thefocal distance of the imaging unit 251 (display object indicating focaldistance, that is, left eye image and right eye image of focus icon).The display controller 234 sets the disparity and convergence angle ofthe focus icon such that a virtual image position of the focus icon is adepth position corresponding to the focal distance. The displaycontroller 234 causes the display image processor 232 to apply thesettings to the focus icon. The display controller 234 causes thedisplay unit 112 to display the focus icon to which the disparity andconvergence angle are set such that the depth position is the positioncorresponding to the focal distance.

Note that, at this time, the display controller 234 can use any methodfor setting the disparity and convergence angle of the focus icon. Forexample, Japanese Patent Application Laid-open No. HEI 08-322004discloses a stereoscopic video display apparatus including means forshifting an image electrically displayed on a display surface in ahorizontal direction such that a diopter scale and convergence aremaintained substantially coincident in real time. Further, JapanesePatent Application Laid-open No. HEI 08-211332 discloses a stereoscopicvideo reproducing device that obtains a stereoscopic video using abinocular disparity, the device including a convergence-angle selectionmeans for setting the convergence angle when viewing a reproducing videoand a control means for controlling relative reproducing positions ofleft/right videos based on information related to the selectedconvergence angle. For example, the display controller 234 may set useeither one of these methods for setting the disparity and convergenceangle of the focus icon.

Alternatively, for example, the imaging unit 251 may have an automaticfocal distance adjustment function of adjusting a focal distance withouta user's operation (also referred to as autofocus). For example, thesensor unit 221 may include a distance measurement sensor that measuresa distance to an object of the real space in front of the display unit112 as viewed from the user and the distance measurement sensor maymeasure the distance to the subject imaged by the imaging unit 251.Then, the imaging controller 253 may cause the imaging unit 251 toadjust the focal distance to the distance measured by the distancemeasurement sensor of the sensor unit 221 (distance to subject).

Any distance measurement method for the distance measurement sensor canbe used. For example, a passive type distance measurement techniqueusing a stereo camera or an active type distance measurement techniqueusing a distance image camera or laser camera may be used. For example,Japanese Patent Application Laid-open No. 2012-222386 discloses a methodof specifying a position of a user by a stereo camera. Further, JapanesePatent Application Laid-open No. 2011-205388 discloses a calibrationmethod for a stereo camera. The sensor unit 221 may use either one ofthese methods for measuring the distance to the subject.

Note that the distance measurement sensor may measure a distance to anobject (subject) located at a predetermined position in an area (imagingregion) imaged by the imaging unit 251. For example, the distancemeasurement sensor may measure a distance to a subject in the imagingregion, which is located at a position seen at a center of the displayregion of the display unit 112 as viewed from the user. The sensor unit221 may further include a line-of-sight detection sensor that detects aline-of-sight of the user and the distance measurement sensor maymeasure a distance to an object (subject) in the imaging region, whichis in the line-of-sight direction of the user detected by theline-of-sight detection sensor. In addition, the sensor unit 221 mayinclude a face detection sensor that detects a face portion of a subjectin the imaging region, which is seen in the display region of thedisplay unit 112 as viewed from the user and the distance measurementsensor may measure a distance to the face portion of the subjectdetected by the face detection sensor.

Also in the case where the distance measurement sensor is used tomeasure the distance to the subject and the focal distance is adjustedto the measured distance to the subject as described above (i.e.,autofocus), the display controller 234 sets the disparity andconvergence angle of the focus icon such that a virtual image positionof the focus icon indicating the focal distance is a depth positioncorresponding to the focal distance as in the manual focus. The displayimage processor 232 applies the settings to the focus icon and thedisplay unit 112 displays the thus obtained focus icon.

Note that the face detection sensor may detect face portions of aplurality of subjects in the imaging region, which are seen in thedisplay region of the display unit 112 as viewed from the user and thedistance measurement sensor may measure a distance to the face portionof each subject detected by the face detection sensor. In this case, afocus icon for each subject may be displayed on the display unit 112(i.e., a plurality of focus icons are displayed on the display unit112). That is, any number of focus icons can be displayed on the displayunit 112.

Note that the focus icon can have any shape and may have, for example, across shape, a rectangular shape, or a circular shape or may haveanother shape.

Further, the display controller 234 may set the size of the displayobject according to the focal distance. That is, the focus icondisplayed on the display unit 112 may be displayed with a size accordingto the depth position (i.e., focal distance). For example, the size ofthe focus icon may be reduced as the focal distance increases (in otherwords, the size of the focus icon may be increased as the focal distancedecreases). That is, the focal distance may be expressed using the depthposition and size of the focus icon.

The above-mentioned processing of the respective sections is controlledby the system controller 211. That is, as in the case of the imagingapparatus described in the first embodiment, the system controller 211causes the imaging unit to adjust the focal distance. The systemcontroller 211 causes the image generator to generate the left eye imageand the right eye image of the stereoscopic image such that the displayobject is seen at the depth position corresponding to the focaldistance. The system controller 211 causes the display unit, throughwhich at least a part of the imaging region of the imaging unit can beseen in a see-through manner and which is configured to display thestereoscopic image formed of the left eye image and the right eye image,to display the left eye image and the right eye image of thestereoscopic image. Then, the system controller 211 causes the imagingunit to image the subject.

<Flow of Imaging>

When performing imaging, the system controller 211 controls therespective sections to perform the imaging-related processes asdescribed above. A flow example of imaging will be described withreference to FIG. 3, and also FIGS. 4 to 6 if necessary. In thefollowing description, it is assumed that the imaging unit 251 has anautofocus function, the sensor unit 221 measures a distance to a subjectthrough the distance measurement sensor, and the imaging controller 253causes the imaging unit 251 to adjust the focal distance to the measureddistance.

When the imaging is started, in Step S101, the system controller 211judges whether or not to adjust the focal distance. For example, if thesensor unit 221 receives an instruction to adjust the focal distancefrom the user, the processing of the system controller 211 proceeds toStep S102.

In Step S102, the system controller 211 controls the sensor unit 221 viathe sensor controller 222 to measure the distance to the subject.

In Step S103, the system controller 211 controls the imaging unit 251via the imaging controller 253 to adjust (set) the focal distance to thedistance measured in Step S102. Note that, in the case of manual focus,the processing of Step S102 is omitted and the imaging unit 251 adjusts(sets) the focal distance according to a user's operation in Step S103.

In Step S104, the system controller 211 controls the display controller234 to set the size of the focus icon to a size according to thedistance measured in Step S102. Further, the system controller 211controls the display controller 234 to set the binocular disparity andconvergence angle of the focus icon such that the focus icon isdisplayed at a depth position corresponding to the distance (focaldistance) measured in Step S102.

In Step S105, the system controller 211 controls the image generator 231to generate a stereoscopic image of the focus icon. Then, the systemcontroller 211 controls the display image processor 232 via the displaycontroller 234 to apply the size, the binocular disparity, theconvergence angle, and the like set in Step S104 to the focus icongenerated by the image generator 231, to thereby generate thestereoscopic image of the focus icon with a distance from the user(see-through HMD 100) to a virtual image position (also referred to asvirtual image distance) being the focal distance.

In Step S106, the system controller 211 controls the display drive unit233 and the display unit 112 via the display controller 234 to cause thedisplay unit 112 to display the stereoscopic image of the focus icongenerated in Step S105. That is, the system controller 211 causes theright eye display unit 112A to display a right eye image of the focusicon and causes the left eye display unit 112B to display a left eyeimage of the focus icon.

With this, the focus icon is displayed on the display unit 112 such thatthe user can see the focus icon as if it is present at a position in thereal space, which is at the focal distance from the see-through HMD 100(user).

For example, as shown in “A” of FIG. 4, if a distance from a user 301(see-through HMD 100) to a subject 302 is detected as “a,” the systemcontroller 211 sets the binocular disparity and convergence angle of afocus icon 311 such that the virtual image distance of the focus icon311 displayed on the display unit 112 is also “a.” With this, as shownin “B” of FIG. 4, the focus icon 311 displayed on the display unit 112looks to the user as if it is present at the same position as that ofthe subject 302. That is, the user can easily know that focus isachieved on the subject based on the virtual image position of the focusicon. That is, the user can more easily know the focal distance.

If the subject 302 is displaced frontward by “b” from “a” as shown inFIG. 5, the system controller 211 controls the binocular disparity andconvergence angle of the focus icon 311 to express the virtual imageposition of the focus icon 311 that is also displaced frontward by “b.”At this time, the system controller 211 may control only the virtualimage position of the focus icon 311 (settings of binocular disparityand convergence angle). It should be noted that, in this case, as in theexample of “B” of FIG. 5, the subject 302 is increased in size incomparison with the case of “B” of FIG. 4 while the size of the focusicon 311 is not changed from the size shown in the example of “B” ofFIG. 4. In other words, the size relationship between the subject 302and the focus icon 311 is changed (the focus icon 311 becomes smaller incomparison with the subject 302). It may make the user uncomfortable.That is, the user may feel as if the focus icon 311 becomes fartherbecause the focus icon 311 becomes smaller, though the virtual imageposition of the focus icon 311 becomes closer.

In view of this, the system controller 211 may change the size of thefocus icon 311 displayed on the display unit 112 according to thevirtual image position of the focus icon 311 as in the example of “C” ofFIG. 5. For example, as the virtual image position of the focus icon 311becomes closer (the virtual image distance decreases), the focus icon311 may be displayed with a larger size. Note that the amount of changeof the focus icon 311 may be proportional to the amount of change of thevirtual image position or the amount of change may be changed dependingon the virtual image position to emphasize the change in size. Forexample, as the virtual image position becomes closer, the amount ofchange in size of the focus icon 311 may be increased.

Further, as shown in “A” of FIG. 6, if the subject 302 is displacedrearward by “c” from “a,” the system controller 211 controls thebinocular disparity and convergence angle of the focus icon 311 toexpress the virtual image position of the focus icon 311 that is alsodisplaced rearward by “c.” At this time, the system controller 211 maycontrol only the virtual image position of the focus icon 311 (settingsof binocular disparity and convergence angle). It should be noted that,in this case, as in the example of “B” of FIG. 6, the subject 302becomes smaller than that in the case of “B” of FIG. 4 while the size ofthe focus icon 311 is not changed from the size shown in the example of“B” of FIG. 4. In other words, the size relationship between the subject302 and the focus icon 311 is changed (the focus icon 311 is increasedin size in comparison with the subject 302). It may make the useruncomfortable. That is, the user may feel the focus icon 311 closerbecause the focus icon 311 becomes larger, though the virtual imageposition of the focus icon 311 becomes farther.

In view of this, as in the example of “C” of FIG. 6, the systemcontroller 211 may change the size of the focus icon 311 displayed onthe display unit 112 according to the virtual image position of thefocus icon 311. For example, as the virtual image position of the focusicon 311 becomes farther (the virtual image distance increases in size),the focus icon 311 may be displayed with a smaller size. Note that theamount of change in size of the focus icon 311 may be proportional tothe amount of change in the virtual image position (constantirrespective of the virtual image position) or the amount of change maybe changed depending on the virtual image position for emphasizing thechange in size. For example, as the virtual image position becomesfarther, the amount of change in size of the focus icon 311 may bedecreased.

Note that the virtual image distance (virtual image position) of thefocus icon 311 can be changed by the system controller 211 changing thedisplay position of the focus icon 311 in each of the right eye imageand the left eye image. For example, it is assumed that the convergenceangle when the virtual image distance is indicated by “a” is α as in “A”of FIG. 4, the convergence angle when the subject 302 becomes closer tothe user by “b” from the virtual image distance “a” is β as in “A” ofFIG. 5, the convergence angle when the subject 302 becomes farther fromthe user by “c” from the virtual image distance “a” as in “A” of FIG. 6is γ, and a distance between the left and right pupils is D. Providedthat D=61.5 mm and “a”=4000 mm, α=53 minutes.

Provided that an amount corresponding to one pixel in the display unit112 (each of the right eye display unit 112A and the left eye displayunit 112B) is three minutes, when the display position of the focus icon311 is displaced from the predetermined position horizontally inwards byan amount corresponding to one pixel, γ=56 minutes and “b”=225 mm.

Further, when the display position of the focus icon 311 in the displayunit 112 (each of the right eye display unit 112A and the left eyedisplay unit 112B) is displaced from the predetermined positionhorizontally outwards by an amount corresponding to one pixel, γ=50minutes and c=228 mm.

As described above, the convergence angle is changed by changing thedisplay position of the focus icon, and hence the virtual image distancecan be changed to any distance.

For example, if the system controller 211 does not completely adjust thevirtual image distance of the focus icon 311 to the distance between theuser 301 (see-through HMD 100) and the subject 302 that is measured indistance measurement, as long as the front and rear relationship betweenthe focus icon 311 and the subject 302 is constant, it helps the user toknow a focused position. Thus, also in this case, the same effects canbe provided.

Note that, for example, after the focal distance adjustment, forexample, if the user 301 changing the direction of the see-through HMD100 (i.e., imaging direction) and an object of the real space ispositioned in front of the position at which the focus icon 311 is seento be present in the display region of the display unit 112, the systemcontroller 211 may be configured not to display a portion of the focusicon 311, which overlaps with the object for expressing the front andrear relationship. For example, if the positional relationship betweenthe focus icon 311 and the object of the real space becomes arelationship in which the focus icon 311 is entirely hidden by theobject, the system controller 211 may be configured not to display thefocus icon 311.

Referring back to FIG. 3, if the processing of Step S106 ends, theprocessing proceeds to Step S107. Further, if it is judged in Step S101that the focal distance is not to be adjusted because, for example, theinstruction is not received from the user, the processing proceeds toStep S107.

In Step S107, the system controller 211 judges whether or not to imagethe subject. For example, if the sensor unit 221 receives an instructionto image the subject from the user, the processing of the systemcontroller 211 proceeds to Step S108.

In Step S108, the system controller 211 controls the imaging unit 251via the imaging controller 253 to image the subject and generates imagedata of the captured image.

When the image data of the captured image is obtained, the processingproceeds to Step S109. Further, if it is judged in Step S107 that thesubject is not to be imaged because, for example, the sensor unit 221does not receive the instruction to image the subject from the user, andthe processing of the system controller 211 proceeds to Step S109.

In Step S109, the system controller 211 judges whether or not toterminate the imaging. For example, if it is judged that imaging is tobe continued because the sensor unit 221 does not receive an instructionto terminate imaging from the user or the like, the processing of thesystem controller 211 returns to Step S101 and the subsequent processingis performed.

Further, if it is judged in Step S109 that the imaging is to beterminated because, for example, the sensor unit 221 receives theinstruction to terminate the imaging from the user or the like, thesystem controller 211 terminates the imaging.

By performing the imaging in the above-mentioned manner, the see-throughHMD 100 (system controller 211) is capable of controlling the respectivesections to control the binocular disparity and convergence angle suchthat the virtual image distance of the stereoscopic image of the focusicon indicating the focal distance is adjusted to the focal distance,and causing the display unit 112 to display it. With this, the user canmore easily know the focal distance.

<Specific Example of Focal Distance Adjustment>

Next, how to adjust the focal distance will be described as an example.The position of the focus icon 311 may be fixed to the predeterminedposition (e.g., center) in the display region of the display unit 112.FIG. 7 shows the example.

For example, as in “A” of FIG. 7, it is assumed that there are subjects321 to 323 that are objects of the real space in the display region ofthe display unit 112. By the user inputting an instruction, for example,the camera function is activated. Then, as shown in “B” of FIG. 7, forexample, the focus icon 311 is displayed at the center of the displayregion of the display unit 112.

As shown in “C” of FIG. 7, the user changes the direction of thesee-through HMD 100 (imaging direction) by changing the direction of theface such that the focus icon 311 is overlapped with the desired subject322.

In this state, the measurement of the focal distance is executed andthen a distance to the subject 322 is measured. The binocular disparityand convergence angle of the focus icon 311 and the size of the focusicon 311 are adjusted such that the virtual image distance of the focusicon 311 coincides with the measured distance to the subject 322. Withthis, as shown in “C” of FIG. 7, the virtual image position and size ofthe focus icon 311 displayed on the display unit 112 are changeddepending on the distance to the subject 322 and the user is informed ofthe fact that focus is achieved on the subject 322. At this time, asound feedback may be given.

In this state, the user or the like instructs to capture an image, andthen the image is captured and image data of the captured image isobtained. The execution instruction of the focal distance adjustment orimaging as described above are input by the user touching thesee-through HMD 100 or a device relating to an imaging function thereof,operating a multi-stage press button, or uttering voice. Note that thefocal distance may be adjusted and the camera function may be activatedas processes before the subject is imaged.

Note that an instruction input of the user or the like is necessary fora trigger of the focal distance adjustment, from the state of “C” ofFIG. 7, as in “D” of FIG. 7, even when the user changes the direction ofthe see-through HMD (imaging direction) by changing the direction of theface and displaces the focus icon 311 from the subject 322, the focaldistance is maintained. That is, also in the state of “D” of FIG. 7,focus is achieved on the subject 322.

With this configuration, the user can more freely set the composition ofthe captured image while more easily maintaining the focal distance.

Note that the focus icon 311 can have any shape. For example, as shownin “E” of FIG. 7, the focus icon 311 may be rectangular. Further, forexample, as shown in “F” of FIG. 7, not only the focus icon 311 but alsoa rectangular frame 331 that is an image showing the field angle of thecaptured image may be displayed on the display unit 112. Note that thisimage does not need to be the stereoscopic image.

Further, the display position of the focus icon 311 in the displayregion may be variable. For example, the display position of the focusicon 311 (position of object as target to which focal distance isadjusted) may be linked with the line-of-sight of the user (focus isachieved on subject in direction of line-of-sight of user). An examplethereof will be shown in FIG. 8.

For example, as shown in “A” of FIG. 8, it is assumed that there aresubjects 321 to 323 that are objects of the real space in the displayregion of the display unit 112 as in “A” of FIG. 7. By the userinputting an instruction, for example, the camera function is activated.Then, as shown in “B” of FIG. 8, for example, the focus icon 311 isdisplayed at the center of the display region of the display unit 112.

In this state, as shown in “C” of FIG. 8, the user moves theline-of-sight. Then, the sensor unit 221 detects the line-of-sight andthe focus icon 311 is moved to the position corresponding to theline-of-sight. In the example of “C” of FIG. 8, the user directs theline-of-sight to the subject 322. Therefore, the focus icon 311 is movedto a position overlapping with the subject 322.

In this state, when the measurement of the focal distance is executed,as in the example of FIG. 7, a distance to the subject 322 is measured.The binocular disparity and convergence angle of the focus icon 311 andthe size of the focus icon 311 are adjusted such that the virtual imagedistance of the focus icon 311 coincide with the measured distance tothe subject 322. With this, as shown in D of FIG. 8, the virtual imageposition and size of the focus icon 311 displayed on the display unit112 are changed to a distance to the subject 322 and the user isinformed of the fact that focus is achieved on the subject 322. At thistime, a sound feedback may be given.

With this configuration, the user can more freely set the composition ofthe captured image while more easily maintaining the focal distance.

Of course, also in the example of FIG. 8, as in FIG. 7, even if thedirection of the line-of-sight is changed, the focal distance may befixed unless the user or the like inputs an instruction of the focaldistance adjustment. Also in this case, the focus icon 311 can have anyshape. Further, for example, not only the focus icon 311 but also arectangular frame 311 that is an image showing the field angle of thecaptured image may be displayed on the display unit 112. Note that thisimage does not need to be the stereoscopic image.

Although the display unit 112 is the see-through display that is lighttransmissive has been described above, the display unit 112 may be anon-see-through display that is not light transmissive. For example, onthe display unit 112 (non-see-through display), the display object(focus icon) may be displayed overlapped on a captured image (alsoreferred to as through-image) obtained by the imaging unit 251. Also inthis case, as in the above-mentioned see-through display, it is onlynecessary to adjust the binocular disparity, convergence angle, size,and the like such that the virtual image position of the display object(focus icon) is a position corresponding to the focal distance. That is,the present technology is applicable not only to the see-through HMD 100but also to a non-see-through HMD, for example.

The above-mentioned series of processing may be executed by hardware ormay be executed by software. If the above-mentioned series of processingis executed by software, programs configuring the software is installedfrom a network or a storage medium.

The storage medium is, in addition to the apparatus main body,configured by, for example, the removable medium 264 in which theprograms are stored, which is distributed for delivering the programs tothe user as shown in FIG. 2. The removable medium 264 includes amagnetic disc (including flexible disc) and an optical disc (includingCD-ROM and DVD). In addition, the removable medium 264 includes amagneto-optical disc (including mini disc (MD)), a semiconductor memory,and the like.

In this case, the programs can be installed into the storage unit 261 bya removable medium thereof being mounted on a drive.

Further, the programs can also be provided via a wired or wirelesstransmission medium such as a local area network, the Internet, anddigital terrestrial broadcasting. In this case, the programs can bereceived by the communication unit 262 of each apparatus and installedinto the storage unit 261.

The programs may be installed into the ROM of the system controller 211or the storage unit 261 in advance.

Note that the programs executed by the computer may be programs forperforming processing in a time series in the order described herein ormay be programs for performing processing in parallel or at a necessarytiming, for example, when called.

Further, a step of describing the programs stored in the storage mediumincludes processes performed in a time series in the order describedherein, of course. However, the processes do not necessarily need to beprocessed in a time series and may be performed in parallel orindividually.

Further, the processes of the above-mentioned steps can be performed byeach of the above-mentioned apparatuses or any apparatus other than theabove-mentioned apparatuses. In this case, the apparatus that performsthe processes only needs to have the above-mentioned functions(functional blocks, etc.) necessary for performing the processes.Further, it is only necessary to appropriately transmit informationnecessary for the processes to the apparatus.

Further, in the specification, the system means a collection of aplurality of components (apparatuses, modules (parts), etc.) and all thecomponents do not necessarily need to be in the same casing. Thus, aplurality of apparatuses housed in individual casings and connected viaa network and a single apparatus including a plurality of modules housedin one casing are both the system.

Alternatively, the component described above as a single apparatus (orprocessors) may be divided and configured as a plurality of apparatuses(or processors). In contrast, the components described as a plurality ofapparatuses (or processors) described above may be collected andconfigured as a single apparatus (or processors). Further, a componentother than those described above may be added to the components of eachapparatus (or each processor). In addition, as long as substantially thesame components and operations are provided in the entire system, someof the components of a certain apparatus (or processor) may be includedin other apparatuses (or other processors).

Although favorable embodiments of the present disclosure have beendescribed above in details with reference to the accompanied drawings,the technical range of the present disclosure is not limited to theabove-mentioned examples. It is clear that those skilled in the art canachieve various changed or modified examples without departing from thetechnical concept defined by the scope of claims. Of course, it shouldbe understood that the changed or modified examples fall in thetechnical range of the present disclosure.

For example, the present technology can take a cloud computingconfiguration in which a single function is shared and processed by aplurality of apparatuses via a network.

Further, the steps described above referring to the flowchart can beperformed by a single apparatus or shared and performed by a pluralityof apparatuses.

In addition, if a single step includes a plurality of processes, theplurality of processes of the single step can be performed by a singleapparatus or shared and performed by a plurality of apparatuses.

Further, the present technology is not limited thereto and can becarried out as such an apparatus or any component incorporated in anapparatus configuring the system. For example, the present technologycan be carried out as a processor serving as a system large scaleintegration (LSI), a module using a plurality of processors or the like,a unit using a plurality of modules or the like, or a set obtained byadding other functions (i.e., some components of apparatus) to the unit.

Note that the present technology may also take the followingconfigurations.

-   (1) An imaging apparatus, including:-   an imaging unit;-   a display unit, through which at least a part of an imaging region    of the imaging unit can be seen in a see-through manner and which is    configured to display a stereoscopic image formed of a left eye    image and a right eye image;-   a focal distance adjustment unit configured to adjust-   a focal distance of the imaging unit; and-   a display controller configured to generate the left eye image and    the right eye image such that a display object indicating the focal    distance is seen at a depth position corresponding to the focal    distance and cause the display unit to display the left eye image    and the right eye image.-   (2) The imaging apparatus according to any one of (1) and (3) to    (12), in which-   the display controller is configured to set a size of the display    object according to the focal distance.-   (3) The imaging apparatus according to any one of (1), (2), and (4)    to (12), in which-   the display unit includes-   a left eye image display unit configured to display the left eye    image, and-   a right eye image display unit configured to display the right eye    image.-   (4) The imaging apparatus according to any one of (1) to (3) and (5)    to (12), in which-   the imaging apparatus includes a casing that is mounted on a head of    a user such that the left eye image display unit is positioned near    in front of a left eye of the user and the right eye image display    unit is positioned near in front of a right eye of the user.-   (5) The imaging apparatus according to any one of (1) to (4) and (6)    to (12), in which-   the display controller is configured to cause the display unit to    further display an image indicating an imaging field angle of the    imaging unit.-   (6) The imaging apparatus according to any one of (1) to (5) and (7)    to (12), in which-   the focal distance adjustment unit is configured to be operated by a    user to adjust the focal distance.-   (7) The imaging apparatus according to any one of (1) to (6) and (8)    to (12), further including-   a distance measurement unit configured to measure a distance to a    subject, in which-   the focal distance adjustment unit is configured to adjust the focal    distance to the distance measured by the distance measurement unit.-   (8) The imaging apparatus according to any one of (1) to (7) and (9)    to (12), in which-   the distance measurement unit is configured to measure a distance to    the subject at a predetermined position in the imaging region.-   (9) The imaging apparatus according to any one of (1) to (8)    and (10) to (12), in which-   the predetermined position is a position in the imaging region,    which is seen at a center of a display region of the display unit.-   (10) The imaging apparatus according to any one of (1) to (9), (11),    and (12), further including-   a face detector configured to detect a face of the subject, in which-   the distance measurement unit is configured to measure a distance to    the face of the subject detected by the face detector.-   (11) The imaging apparatus according to any one of (1) to (10) and    (12), further including-   a line-of-sight detector configured to detect a line-of-sight    direction of the user, in which-   the distance measurement unit is configured to measure a distance to    the subject in the line-of-sight direction of the user detected by    the line-of-sight detector.-   (12) The imaging apparatus according to any one of (1) to (11),    further including-   an instruction reception unit configured to receive an execution    instruction of measurement of the distance to the subject, in which-   the distance measurement unit is configured to measure the distance    to the subject based on the execution instruction received by the    instruction reception unit.-   (13) An imaging method, including:-   adjusting a focal distance of an imaging unit;-   generating a left eye image and a right eye image such that a    display object indicating the focal distance is seen at a depth    position corresponding to the focal distance and causing a display    unit, through which at least a part of an imaging region of the    imaging unit can be seen in a see-through manner and which is    configured to display a stereoscopic image formed of the left eye    image and the right eye image, to display the left eye image and the    right eye image; and-   imaging a subject by the imaging unit.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

What is claimed is:
 1. An imaging apparatus, comprising: an imaging unitconfigured to image a real subject included in an imaging region; adisplay unit configured to display a stereoscopic display image formedof a left eye image and a right eye image in at least one see-throughdisplay; a focal distance adjustment unit configured to adjust a focaldistance of the imaging unit; and a display controller configured tocontrol the display unit to: increase a size of a stereoscopic displayobject as a virtual image position of the stereoscopic display objectapproaches closer to the imaging apparatus, wherein the virtual imageposition corresponds to the focal distance; and at least partiallyoverlap the stereoscopic display object on the real subject when seenthrough the at least one see-through display.
 2. The imaging apparatusaccording to claim 1, wherein the display controller is configured toset the size of the stereoscopic display object according to the focaldistance.
 3. The imaging apparatus according to claim 1, wherein thedisplay unit includes: a left eye image display unit configured todisplay the left eye image, and a right eye image display unitconfigured to display the right eye image.
 4. The imaging apparatusaccording to claim 3, wherein the imaging apparatus includes a casingthat is configured to be mounted on a head of a user such that the lefteye image display unit is positioned near in front of a left eye of theuser and the right eye image display unit is positioned near in front ofa right eye of the user.
 5. The imaging apparatus according to claim 1,wherein the display controller is configured to cause the display unitto further display an image indicating an imaging field angle of theimaging unit.
 6. The imaging apparatus according to claim 1, wherein thefocal distance adjustment unit is configured to be operated by a user toadjust the focal distance.
 7. The imaging apparatus according to claim1, further comprising a distance measurement unit configured to measurea distance to the real subject, wherein the focal distance adjustmentunit is configured to adjust the focal distance to the distance measuredby the distance measurement unit.
 8. The imaging apparatus according toclaim 7, wherein the distance measurement unit is configured to measurea distance to the real subject at a predetermined position in theimaging region.
 9. The imaging apparatus according to claim 8, whereinthe predetermined position is a position in the imaging region, which isseen at a center of a display region of the display unit.
 10. Theimaging apparatus according to claim 7, further comprising a facedetector configured to detect a face of the real subject, wherein thedistance measurement unit is configured to measure a distance to theface of the real subject detected by the face detector.
 11. The imagingapparatus according to claim 7, further comprising a line-of-sightdetector configured to detect a line-of-sight direction of a user,wherein the distance measurement unit is configured to measure adistance to the real subject in the line-of-sight direction of the userdetected by the line-of-sight detector.
 12. The imaging apparatusaccording to claim 7, further comprising an instruction reception unitconfigured to receive an execution instruction of measurement of thedistance to the real subject, wherein the distance measurement unit isconfigured to measure the distance to the real subject based on theexecution instruction received by the instruction reception unit.
 13. Animaging method, comprising: adjusting a focal distance of an imagingunit; increasing a size of a stereoscopic display object as a virtualimage position of the stereoscopic display object approaches closer toan imaging apparatus, wherein the virtual image position corresponds tothe focal distance; displaying a stereoscopic display image formed ofthe left eye image and the right eye image in at least one see-throughdisplay; imaging a real subject by the imaging unit; and at leastpartially overlapping the stereoscopic display object on the realsubject when seen through the at least one see-through display.
 14. Theimaging apparatus according to claim 1, wherein the display unit isconfigured to serially change the size of the stereoscopic displayobject with respect to change of the virtual image position.
 15. Theimaging apparatus according to claim 14, wherein the display unit isconfigured to proportionally increase the size of the stereoscopicdisplay object as the virtual image position of the stereoscopic displayobject approaches closer to the imaging apparatus.